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ABSTRACT 

This  Final  Report  summarizes  work  in  the  above  grant,  which  dealt  with  several  topics 
related  to  the  control  of  nonlinear  systems,  including:  input  to  state  stability  and  new  related 
notions  (such  as  input/output  to  state  stability),  discontinuous  feedback  techniques  based  on 
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1  Introduction 

The  control  of  highly  nonlinear  systems  is  of  central  importance  to  the  Air  Force  mission,  as 
nonlinear  effects  cannot  be  ignored  in  high-performance  aircraft.  This  report  focuses  on  the 
development  of  basic  mathematical  foundations  for  finite-dimensional  nonlinear  control  theory. 

1.1  ISS-Related  Notions 

Methods  of  feedback  design  are  undergoing  an  exceptionally  rich  period  of  progress  and  mat¬ 
uration,  fueled  to  a  great  extent  by  (1)  the  discovery  of  certain  basic  conceptual  notions,  and 
(2)  the  identification  of  classes  of  systems  for  which  systematic  decomposition  approaches  can 
result  in  effective  and  easily  computable  control  laws.  These  two  aspects  are  complementary, 
since  the  latter  approaches  are,  typically,  based  upon  the  inductive  verification  of  the  validity 
of  the  former  system  properties  under  compositions. 

One  of  the  notions  instrumental  in  the  new  developments  is  that  of  input  to  state  stability 
(ISS),  introduced  by  the  PI  in  1989,  Another  one  is  that  of  control- Lyapunov  function  (CLF), 
formalized  and  completely  characterized  in  independent  and  complementary  papers  by  Artstein 
and  the  PI  in  the  early  1980s.  These  notions  have  been  incorporated  into  the  main  current 
books  used  as  texts  and  references  for  advanced  nonlinear  control  courses,  such  as  the  books 
by  Krstic,  M.,  I.  Kanellakopoulos,  and  P.  V.  Kokotovic,  Isidori,  and  Khalil’s  (latest  editions). 

The  ultimate  goal  of  the  Pi’s  work  in  this  area  is  highly  ambitious:  the  complete  reformula¬ 
tion  of  the  foundations  of  nonlinear  control  based  on  ISS-like  ideas.  This  goal  is  very  long-term, 
but  definitely  worth  the  effort:  the  payoff  will  be  the  development  of  a  consistent,  elegant,  and 
ultimately  design-oriented,  systematic  approach  to  the  subject.  After  briefly  surveying  the  con¬ 
cept  of  ISS,  this  report  will  describe  the  dual  notion  of  IOSS  (input /output  to  state  stability), 
which  corresponds  to  “detectability”  of  nonlinear  systems  and  of  IOS  (input  to  output  stability), 
which  should  provide  the  appropriate  setting  for  i/o  behaviors  and  regulation  problems. 

1.2  Discontinuous  Feedback  Design 

It  is  known  that,  in  general,  in  order  to  control  nonlinear  systems  one  must  use  switching 
(discontinuous)  mechanisms  of  various  types.  Of  course,  time-optimal  solutions  for  even  linear 
systems  often  involve  such  discontinuities,  but,  for  linear  systems,  most  control  problems  usually 
admit  also  (perhaps  suboptimal)  continuous  solutions.  However,  when  dealing  with  arbitrary 
systems,  discontinuities  are  unavoidable  even  when  no  optimality  objectives  axe  imposed.  Faced 
with  the  theoretical  impossibility  of  having  feedback  laws  of  the  form  u  =  k(x),  k  a  continuous 
function,  various  alternatives  have  been  proposed.  (Objectives  other  than  state  stabilization, 
such  as  tracking  and  disturbance  rejection,  output  feedback,  or  adaptive  and  robust  variants, 
also  lead  to  the  necessity  of  discontinuous  controllers;  the  study  of  stabilization  is  merely  the 
first  step  in  the  search  for  general  controller  structures.) 

One  way  of  overcoming  the  limitations  of  continuous  static  feedback  is  to  look  for  dynamic 
feedback  laws,  that  is  to  say,  to  allow  for  additional  (memory)  variables  in  controllers.  This 
approach,  suggested  by  the  Pi  and  Sussmann  in  1980,  and  developed  deeply  by  Coron  in  the  late 
1980s,  works  for  certain  restricted  classes  of  systems,  and  is  closely  related  to  the  use  of  classical 
gradient  descent  and  Newton  numerical  algorithms  for  control,  as  shown  in  [8].  However,  in 
general  no  such  dynamic  or  time-varying  solutions  are  known,  so  the  interest  in  discontinuous 
feedback  laws  u  =  k(x)  arises. 
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In  [13],  we  introduced  a  new  formulation  of  discontinuous  feedback,  and,  using  this  no¬ 
tion,  we  showed  that  every  asymptotically  controllable  system  can  be  stabilized.  One  of  the 
contributions  was  in  defining  precisely  the  meaning  of  stabilization  when  the  feedback  rule  is 
not  continuous.  This  work  solved  a  longstanding  open  question  in  nonlinear  control  theory, 
concerning  the  relationship  between  asymptotic  controllability  to  the  origin  and  the  existence 
of  feedback  controls.  The  main  tools  used  were:  (a)  control-Lyapunov  functions,  (b)  methods 
of  nonsmooth  analysis,  especially  proximal  subgradients,  and  (c)  techniques  from  positional 
differential  games. 

1.3  Other  Subjects 

The  Pi’s  invited  presentation  and  paper  at  the  latest  International  Congress  of  Mathemati¬ 
cians  [17]  dealt  with  spaces  of  observables  as  a  unifying  concept  in  nonlinear  control.  The 
systematic  study  of  such  spaces  and  their  properties  allows  a  deep  understanding  of  important 
aspects  of  system  structure,  including  the  relationship  between  input/output  equations  and  re¬ 
alization,  and  the  existence  and  characterization  of  “universal”  controls  of  various  types,  which 
in  turn  form  the  basis  of  numerical  techniques. 

The  paper  [8])  explained  the  Pi’s  “generic  nonsingular  loop”  method  for  the  steering  of 
nonlinear  systems  without  drift.  The  path-planning  problem  for  such  systems  has  been  of  great 
interest  during  the  last  few  years,  and  several  methods,  all  based  on  a  nontrivial  analysis  of  the 
structure  of  the  Lie  algebra  of  vector  fields  generated  by  the  system,  have  been  proposed.  Our 
method  does  not  make  any  structural  assumptions,  and  is  based  on  a  general  transversality 
result  showing  that  generic  controls  are  universally  nonsingulax,  in  the  sense  that  linearizations 
along  the  ensuing  trajectories  axe  controllable,  for  all  initial  states.  Very  recently,  a  student 
at  Rutgers  completed  an  implementation  of  this  technique  (as  a  set  of  MATLAB  M-files),  in  a 
manner  which  allows  easy  generation  of  trajectories  and  with  adaptive  stepsize  improvements. 

1.4  Scope  and  Organization  of  the  report 

Given  the  range  of  subjects  treated,  and  the  different  methodologies  employed  in  each,  lack  of 
space  prevents  giving  details  on  all  subtopics.  Thus  we  have  chosen  only  a  few  selected  subtopics, 
especially  those  represented  in  very  recent  work  and  hence  possibly  less  easily  available  to 
readers.  Overall,  however,  most  of  the  discussion  in  this  report  is  informal,  with  references  to 
the  literature  for  precise  technical  points.  In  addition,  the  Web  page 

http:/ / www.math.rutgers.edu/'  sontag/ 

provides  access  to  a  substantial  number  of  papers  by  the  Pis,  and  these  may  be  consulted  for 
many  more  details  and  a  mathematically  rigorous  presentation. 
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2  Details 

In  this  part  of  the  report,  we  discuss  the  following  subjects: 

•  ISS  and  Related  Stability  Concepts 

•  Nonsmooth  Methods 

•  Universal  Controls 

As  mentioned  earlier,  the  Web  page  http://www.math.rutgers.edu/~sontag/  provides  ac¬ 
cess  to  a  substantial  number  of  papers  by  the  Pis,  to  be  consulted  for  details  and  a  mathemat¬ 
ically  rigorous  presentation. 

2.1  Stability  Notions:  ISS 

We  consider  general  finite-dimensional  systems,  in  the  standard  sense  of  nonlinear  control: 

x(t)  =  f(x(t),u(t)),  y(t)  —  h(x(t)) .  (1) 

(As  usual,  dot  indicates  derivative,  and  one  omits  the  time  argument  t.  To  keep  the  exposition 
elementary,  we  restrict  to  states  in  an  Euclidean  space  Rn ,  though  the  study  of  control  systems 
evolving  in  manifolds  is  not  only  possible  but  more  natural  in  many  applications.) 

There  are  two  conceptually  very  different  ways  to  formulate  the  notion  of  stability  of  control 
systems.  One  of  them,  which  we  may  call  the  input/output  approach ,  relies  on  operator-theoretic 
techniques.  Among  the  main  contributions  to  this  area,  one  may  cite  the  foundational  work 
by  Zames,  Sandberg,  Desoer,  Safanov,  Vidyasagar,  and  others.  In  this  approach,  a  “system” 
is  a  causal  operator  F  =  Fx( o)  :  u(-)  y(*)  between  spaces  of  signals  (for  fixed  initial  states), 

and  “stability”  is  taken  to  mean  that  F  maps  bounded  inputs  into  bounded  outputs,  or  finite- 
energy  inputs  into  finite-energy  outputs.  More  stringent  typical  requirements  in  this  context 
are  that  the  gain  of  F  be  finite  (in  more  classical  mathematical  terms,  that  the  operator  be 
bounded),  or  that  it  have  finite  incremental  gain  (mathematically,  that  it  be  globally  Lipschitz). 
The  input/output  approach  has  been  extremely  successful  in  the  robustness  analysis  of  linear 
systems  subject  to  nonlinear  feedback  and  mild  nonlinear  uncertainties,  and  in  general  in  the 
area  that  revolves  around  the  various  versions  of  the  small-gain  theorem.  Moreover,  geomet¬ 
ric  characterizations  of  robustness  (gap  metric  and  the  like)  are  elegantly  carried  out  in  this 
framework.  Finally,  i/o  stability  provides  a  natural  setting  in  which  to  study  the  classification 
and  parameterization  of  dynamic  controllers. 

On  the  other  hand,  there  is  the  model-based,  or  state-space  approach  to  systems  and  sta¬ 
bility,  where  the  basic  object  is  the  forced  dynamical  system  (1).  In  this  approach,  there  is 
a  standard  notion  of  stability,  namely  Lyapunov  asymptotic  stability  of  the  unforced  system. 
Associated  to  such  a  system,  there  is  the  above-mentioned  operator  F  mapping  inputs  (forcing 
functions)  into  state  trajectories  (or  into  outputs,  if  partial  measurements  on  states  are  of  in¬ 
terest)  .  It  becomes  of  interest  then  to  ask  to  what  extent  Lyapunov-like  stability  notions  for  a 
state-space  system  are  related  to  the  stability,  in  the  senses  discussed  in  the  previous  paragraph, 
of  the  associated  operator  F.  It  is  well-known  that,  in  contrast  to  the  case  of  linear  systems, 
where  there  is  -subject  to  mild  technical  assumptions-  an  equivalence  between  state-space  and 
i/o  stability,  for  nonlinear  systems  the  two  types  of  properties  are  not  so  closely  related.  Even 
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for  the  very  special  and  comparatively  simple  case  of  “feedback  linearizable”  systems,  this  re¬ 
lation  is  far  more  subtle  than  it  might  appear  at  first  sight:  if  one  first  linearizes  a  system 
and  then  stabilizes  the  equivalent  linearization,  in  terms  of  the  original  system  one  does  not  in 
general  obtain  a  closed-loop  system  that  is  input/output  stable  in  any  reasonable  sense. 

This  leads  one  to  focus  on  the  study  of  the  dependence  of  state  trajectories  on  the  size  of 
inputs,  a  study  which  is  especially  relevant  when  the  inputs  in  question  represent  disturbances 
acting  on  a  system.  For  not  necessarily  linear  systems,  there  is  no  complete  agreement  as 
yet  regarding  what  are  the  most  useful  formulations  of  system  stability  with  respect  to  input 
perturbations.  (For  linear  systems,  similar  considerations  led  to  the  development  of  gains 
and  the  operator-theoretic  approach,  including  the  formulation,  when  using  L2  norms,  of  H°° 
control.)  One  candidate  for  such  a  formulation  is  the  property  called  “input  to  state  stability” 
(ISS),  introduced  in  a  1989  paper  by  the  PI.  This  property  has  proved  to  be  a  very  useful 
paradigm  in  the  study  of  nonlinear  stability  for  systems  subject  to  external  effects,  as  evidenced 
by  the  wide  number  of  current  papers  and  textbooks  that  use  the  notion  in  a  fundamental 
manner.  The  notion  differs  fundamentally  from  the  operator-theoretic  ones,  first  of  all  because 
it  takes  account  of  initial  states  in  a  manner  fully  compatible  with  Lyapunov  stability.  Second, 
boundedness  (finite  gain)  is  far  too  strong  a  requirement  for  general  nonlinear  operators,  and 
it  must  be  replaced  by  “nonlinear  gain  estimates,”  in  which  the  norms  of  output  signals  are 
bounded  by  a  nonlinear  function  of  the  norms  of  inputs;  the  definition  of  ISS  incorporates  such 
gains  in  a  natural  way.  In  very  informal  terms,  the  ISS  property  translates  into  the  statement 
that  “no  matter  what  is  the  initial  state,  if  the  inputs  are  small,  then  the  state  must  eventually 
be  small”  (the  precise  definition  is  recalled  below). 

The  ISS  notion  was  originally  introduced  in  a  1989  paper  by  the  PI  and  has  since  been 
employed  by  several  authors  in  deriving  results  on  control  of  nonlinear  systems.  It  can  be 
stated  in  several  equivalent  manners,  which  indicates  that  it  is  a  mathematically  natural  con¬ 
cept:  dissipation,  robustness  margins,  and  classical  Lyapunov-like  definitions.  The  dissipation 
characterizations  are  closely  related  to  the  pioneering  work  of  Willems  and  Hill  and  Moylan, 
who  introduced  an  abstract  concept  of  energy  dissipation  in  order  to  unify  i/o  and  state  space 
stability,  and  in  particular  with  the  purpose  of  understanding  conceptually  the  meaning  of 
Kalman- Yakubovich  positive-realness  (passivity),  and  frequency-domain  stability  theorems  in 
a  general  nonlinear  context. 

ISS:  Some  technical  details* 

We  review  here  some  of  the  main  characterizations  of  the  ISS  property.  Euclidean  norm  in  Rn  or  Rm  is 
denoted  simply  as  |-|.  (Often,  for  instance  in  the  context  of  studying  stability  of  invariant  sets,  or  when 
studying  so-called  “practical  stability” ,  one  defines  properties  relative  to  a  subset  A  of  M” ;  given  such  a 
set  A ,  one  may  define  instead  |x|  =  d  (x,  A),  and  similar  results  are  proved.)  The  map  /  :  Rn  x  Mm  — ¥  Rn 
in  (1)  is  assumed  to  be  locally  Lipschitz  continuous,  and  /(0,0)  =  0,  h(0)  =  0. 

By  a  control  or  input  we  mean  a  measurable  and  locally  essentially  bounded  function  u  :  R>o  -4  Rm  • 
Given  any  control  u  and  any  (elf,  there  is  a  unique  maximal  solution  of  the  initial  value  problem 
x  =  f(x,u),  x(Q)  =  £,  denoted  by  x(-,£,u).  The  trajectory  x(t,£,u),  and  consequently  also  y(t,£,u)  = 
h(x(t,£,u)),  are  defined  on  some  maximal  interval  [0,tmax),  where  tmax  =  tm ax(£)  <  +oo.  The  1/^,-norm 
(possibly  infinite)  of  a  control  u  is  denoted  by  ||u||. 

A  function  F  :  S  -4  R  defined  on  a  subset  S  of  R"  containing  0  is  said  to  be  positive  definite  if 
F(x)  >  0  for  all  x  €  5,  x  0,  and  F(0)  =  0,  and  it  is  proper  if  the  preimage  F~l(-D,D)  is  bounded, 
for  each  D  >  0.  A  function  7  :  R>0  -4  R>0  is  of  class  M  if  it  is  continuous  and  nondecreasing;  it  is  of 


‘this  subsection  can  be  skipped  without  loss  of  continuity. 
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class  A f0  if  in  addition  it  satisfies  7(0)  =  0,  of  class  K  if  it  is  continuous,  positive  definite,  and  strictly 
increasing,  and  of  class  K0 0  if  it  is  also  unbounded.  Finally,  /? :  R>o  x  R>o  “>  M>o  is  said  to  be  a  function 
of  class  ICC  if  for  each  fixed  t  >  0,  /?(•,£)  is  of  class  K  and  for  each  fixed  s  >  0,  /?(s,t)  decreases  to  zero 
as  t  — ►  00. 

For  simplicity  of  statements,  we  assume  here  that  the  system  is  forward  complete,  that  is  to  say,  for 
each  input  and  each  f,  the  solution  x(t}£,u)  is  defined  on  the  entire  interval  R>o- 

We  define  next  several  natural  properties  of  control  systems  and  characterize  ISS  in  alternative 
manners.  The  original  definition  of  (ISS)  given  in  the  1989  paper  by  the  PI  was  as  follows: 

3  7  €  /C,  /3  <E  KC  st :  Rn  Vu(-)  Vt  >  0 

!*(*>& «)l  <  max{/3(^|,i),7(||«||)}  .  (2) 

(A  sum  can  be  used  instead  of  “max”  and  an  equivalent  concept  results.)  It  was  shown  in  [4]  that  a 
system  is  ISS  if  and  only  if  it  satisfies  a  dissipation  inequality,  that  is  to  say,  there  exists  a  smooth 
V  :  Rn  -4  M>o  (an  “ISS-Lyapunov  function”)  and  there  are  functions  (XiZKoo,  i  =  1,2,3  and  aefC  so 
that 

ai(lfl)<V(0<«a(KI)  (3) 

and 

W(0/(e,«)<<r(H)-as(|f|)  (4) 

for  each  £ERn  and  v€Mm.  A  very  useful  modification  of  this  characterization  due  to  Praly  and  the  PFs 
former  student  Wang  is  the  fact  that  the  ISS  property  is  also  equivalent  to  the  existence  of  a  smooth  V 
satisfying  (3)  and  Equation  (4)  replaced  by  an  estimate  of  the  type 

W(0/(*,ti)<-V(  fl-a3(M). 

This  can  be  understood  as:  “for  some  positive  definite  and  proper  functions  y  =  V(x)  and  v  =  W(u)  of 
states  and  outputs  respectively,  along  all  trajectories  of  the  system  we  have  y  =  —  y  +  vn . 

If  all  trajectories  of  this  system  with  initial  conditions  satisfying  |rc(0)|  <  p,  for  some  p  >  0,  satisfy 
limt-too  |x(t,^,0)|  ->  0,  where  0  denotes  the  control  which  is  identically  equal  to  zero  on  K>o,  one  says 
that  the  system  has  the  0-local  attraction  property.  The  0-local  stability  property  means  that  for  each 
e  >  0  there  is  a  S  >  0  so  that  |f|  <  S  implies  that  |z(£,f,0)|  <  e  for  all  t  >  0.  The  conjunction  of  these 
two  properties  is  the  usual  local  asymptotic  stability  (0-AS).  Next  we  introduce  several  new  concepts. 
The  limit  property  holds  if  every  trajectory  must  at  some  time  get  to  a  ball  around  the  origin  which  is 
a  function  of  the  magnitude  of  the  input: 

3 7  €  Af0  st :  V £  €  Rn  Vu(-) 

inf|z(t,(,«)|  <  7  (INI)  •  (LIM) 

The  asymptotic  gain  property  holds  if  every  trajectory  must  ultimately  stay  not  far  from  zero,  depending 
on  the  magnitude  of  the  input: 

37GA/r0st:Vf  eRn  Vti(-) 

jHmJa;(t)£,u)|  <  7  (INI)  •  (AG) 

The  uniform  asymptotic  gain  property  holds  if  the  above  limsup  is  attained  uniformly  with  respect  to 
initial  states  in  compacts  and  all  u : 

37  €  Af0  Ve  >  0  Vk>0  3T  =  T(e,*0  >0st:  V|f|  <k 

sup|x(t,^,w)|  <  7(||u||)+£  Vu(-)  •  (UAG) 

*>T 

The  boundedness  property  holds  if  bounded  initial  states  and  controls  produce  uniformly  bounded  tra¬ 
jectories: 

3^1^26  AT  st:  V£  €  Mn  Vu(-) 
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sup|z(t,f,u)|  <  max{(Ti(|C|) ,  <t2(||u||)}  .  (BND) 

t>o 

(This  is  sometimes  called  the  “UBIBS”  or  “uniform  bounded-input  bounded-state”  property.)  The 
global  stability  property  holds  if  in  addition  small  initial  states  and  controls  produce  uniformly  small 
trajectories: 

3ai,&2  £  A/o  st :  V£  G  Kn  V it(*) 

sup|x(t,f,u)|  <  max{<ri(|£|) ,  ff2(||u||) }  •  (GS) 

t>  o 

It  is  shown  in  [12]  that  the  ISS  property  is  equivalent  to  each  of  the  following: 

(LIM)  and  (0-AS) 

(AG)  and  (GS) 

(UAG) 

(LIM)  and  (GS) 

(Several  more  equivalences  are  provided  as  well.)  It  is  remarkable  that  several  of  the  properties  do  not 
involve  uniformity  with  respect  to  inputs;  since  no  linearity  assumptions  are  made  regarding  the  depen¬ 
dence  of  the  right  hand  side  on  inputs,  there  is  no  obvious  way  to  use  directly  any  sort  of  compactness 
(e.g.,  in  the  weak  topology),  so  the  proofs  are  quite  delicate. 

2.2  Stability  Notions:  IOSS 

Given  the  central  role  often  played  in  control  theory  by  the  duality  between  input/state  and 
state/output  behavior,  one  may  reasonably  ask  what  concept  obtains  if  outputs  are  used  instead 
of  inputs  in  the  ISS  definition  (2).  This  corresponds  roughly  to  asking  that  “no  matter  the  initial 
state,  if  the  outputs  are  small,  then  the  state  must  be  eventually  small” .  For  linear  systems, 
the  notion  that  arises  is  that  of  detectability.  Thus,  it  would  appear  that  this  dual  property, 
which  we  called  in  [14]  output  to  state  stability  (OSS),  is  a  natural  candidate  as  a  concept  of 
nonlinear  detectability. 

Using  a  naive  dualization  of  the  dissipation  characterization  (4),  one  could  also  ask  what 
is  the  notion  that  emerges  when  we  ask  that  there  be  some  function  V  such  that,  along  all 
possible  trajectories,  V  decreases  if  the  outputs  are  not  too  large  compared  to  the  present  states. 
For  linear  systems,  one  obtains  again  detectability.  For  nonlinear  systems,  variations  of  this 
property  have  very  often  been  suggested  as  a  notion  of  detectability  as  well,  as  mentioned  later. 
It  is  easy  to  see  that  this  dissipation  property,  once  rigorously  formulated,  implies  OSS.  The 
main  contribution  of  [14]  was  to  prove  a  certain  converse  implication.  However,  the  converse 
proof  does  not  provide  a  smooth  V,  which  is  required  for  “backstepping”  and  other  design 
arguments.  (We  conjecture  that  the  full  converse  (smooth  V)  holds.)  We  believe  that  this  will 
be  a  most  useful  foundational  result  for  further  research  in  input/output  stability,  probably  as 
useful  as  those  involving  ISS.  Combining  ISS  and  OSS  there  results  the  notion  of  input/output- 
to-state  stability  (IOSS),  whose  study  was  also  started  in  that  paper.  In  this  case,  we  have  only 
proved  the  easy  implication  (dissipation  property  implies  IOSS)  but  we  conjecture  that  the 
converse  holds  as  well.  The  notion  of  IOSS  is  closely  connected  to  the  possibility  of  stabilizing 
a  partially  observed  system  using  only  output  measurements,  and  is  implied  by  the  existence 
of  observers.  It  is  also  implied  by  strict  passivity.  (Some  further  details  are  provided  below.) 

The  IOSS  property  is  connected  with  the  possibility  of  stabilization  by  output  feedback, 
and  also  to  the  existence  of  (one-sided)  estimators  of  the  norm  of  the  state  (see  below).  These 
are  essentially  properties  connected  with  “zero-detectability” ,  meaning  being  able  to  asymptot¬ 
ically  distinguish  any  given  state  f  from  the  zero  state.  For  linear  systems,  this  O-detectability 
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property  is  equivalent  to  detectability:  being  able  to  asymptotically  distinguish  every  pair  of 
states.  But  for  general,  nonlinear,  systems,  these  properties  are  very  different.  Estimates  of 
arbitrary  states  are  not  necessarily  required  if  the  objective  is  merely  to  drive  the  state  to  the 
special  state  “zero” . 

One  possible  definition  of  “observer”  for  (1)  is  that  of  a  dynamical  system  which  processes 
inputs  and  outputs  of  (1)  and  produces  an  estimate  £(£)  of  the  state  x(t).  The  estimation 
condition  would  be  that  x(t)  -  £(<)  -4  0  as  t  *4  oo  and  that  this  difference  (the  estimation 
error)  should  be  small  if  it  starts  small  However,  it  is  far  more  natural  in  the  context  of  ISS- 
type  notions  to  require  that  the  estimation  error  x(t)  —  x(t)  be  small  even  if  the  measurements 
of  inputs  and  outputs  taken  by  the  observer  are  “noisy”.  Writing  u d  and  y for  the  input 
and  output  measurement  noises,  we  have  the  situation  in  Figure  1,  which  is  formalized  in  the 
technical-details  section  to  follow,  in  the  special  case  of  full  state  observers. 

ud(t) 


I 


Vd(t) 


Figure  1:  Observer  with  noise  and  y& 

The  ISS-style  definition  insures  that  the  error  x(t)  —  z(t)  converges  to  zero  when  there  is  no 
noise  in  the  measurements  taken  by  the  observer,  but  also  that  in  general  it  degrades  gracefully 
as  a  function  of  the  magnitude  of  such  disturbances.  This  seems  most  natural,  and  we  intend 
to  seriously  study  the  implications  of  this  definition. 

IOSS:  Some  technical  details* 

The  system  (1)  is  input/ output-to- state  stable  (IOSS)  if  there  exist  some  /3  €  ICC  and  71,72  €  /C  such 
that 

|s(t>  u)l  <  rnax{/?(|£| ,  t),  71  (||u|[0,t]||)  ,  72  (||i/€,u|[o,t]||)}  (5) 

for  every  initial  state  £  and  control  u  and  all  t  G  [0,tmax))  tmax  =  tmSLX(^u).  As  in  page  4,  z(t,  £,  u) 
denotes  the  trajectory  that  results  from  initial  state  £  and  input  u,  and  y^,u(t)  =  y{t ,  £,  u)  =  h(x(t ,  £,  u)). 

This  property  has  appeared  before  in  the  literature.  It  represents  a  natural  combination  of  the 
notions  of  “strong”  observability  and  ISS.  When  there  are  no  controls,  that  is,  we  are  dealing  with 
a  system  x  =  f(x),y  =  h(x),  we  call  this  property  simply  output  to  state  stability  (OSS).  Note  the 
formal  duality  with  ISS:  we  have  an  estimate  |:r(£, £)|  <  max  {/?(|£| ,  t))  7  (||2/sl[o,t]||)}  compared  to 
|x(t,  £,  u)\  <  max  {/?(|£|  >  t ),  7  (||w|[0>t]  ||) }  •  (the  restriction  of  the  control  makes  no  difference,  by  causal¬ 
ity).  Unfortunately,  this  apparent  duality  does  not  seem  to  help  at  all  in  proving  results  regarding  OSS 
starting  from  ISS  results  (there  is  no  “semigroup  property”  for  states  with  respect  to  outputs). 

An  IOSS- Lyapunov  function  for  system  (1)  is  any  function  V  so  that  there  exist  /Coo-functions  a\ 
and  (*2  such  that 

_ <*i(l£l)  <V(0  <  aa(|£|),  V£  €  IT,  (6) 

"this  subsection  can  be  skipped  without  loss  of  continuity. 
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V  is  absolutely  continuous  along  trajectories,  for  all  initial  states  and  controls,  and  there  exist  JC oo- 
functions  a,  01,  and  oi  such  that  for  every  trajectory  x(t,  £,  u),  and  almost  all  t  e  [0,  <max), 

^V(x(t,{,u))  <  -a(|x(t,  £,  u)|)  +  ai(|u(t)|)  +  <72(|j/(t,^,u)|).  (7) 

In  the  very  recent  paper  [14]  and  ongoing  work,  we  have  shown  that  for  systems  with  no  controls  the 
existence  of  an  IOSS-Lyapunov  function  is  equivalent  to  the  IOSS  property,  and  a  paper  in  preparation 
extends  to  the  full  problem.  As  mentioned  earlier,  the  OSS  property  can  be  thought  of  as  a  definition 
of  detectability.  Indeed,  variations  of  this  notion  can  be  found  at  various  places  in  the  literature.  The 
definition  involving  comparison  functions  implies  in  particular  that  the  system  is  “zero  detectable” .  This 
means  that  under  zero  inputs,  states  whose  outputs  are  identically  zero  should  form  an  asymptotically 
stable  subsystem  (that  is,  y  =  0  implies  x  0,  plus  a  local  stability  condition).  That  definition  relates 
to  the  current  definition  (which  says  in  addition  that  y  0  implies  x  -»  0  and  that  y  small  implies  x 
small)  in  exactly  the  same  way  that  global  asymptotic  stability  of  an  unperturbed  system  x  =  /(x,0) 
relates  to  the  ISS  property.  The  definition  which  involves  Lyapunov  functions  had  also  appeared  in 
restricted  forms.  In  several  papers,  one  finds  detectability  defined  by  the  requirement  that  there  should 
exist  a  (differentiable)  storage  function  V  satisfying  our  Equations  (6)  and  (7),  but  with  the  special 
choice  a(y)  —  |y|2.  A  variation  of  this  is  to  weaken  (7)  to  require  merely 

x^O  =>  ^.V(x(t,Q)  <a(\y(t,0\) 

as  done  for  instance  in  the  definition  of  detectability  given  in  recent  work  by  Morse. 

There  is  another  type  of  relationship  between  the  notions  of  OSS  and  ISS,  different  from  the  obvious 
formal  duality.  Consider  a  system  in  cascade  form 


Xi  =  f(x  i,x2) 

X2  =  tf(x2) 

where  n  =  ni  +  n2  and  the  variables  Xi  have  sizes  n  1  and  n 2  respectively.  Assume  that  the  output 
is  y  =  X2-  If  we  interpret  the  ^-dimensional  subsystem  i  =  g{z)  as  a  generator  of  input  signals 
(“exosystem”)  for  the  m-dimensional  system  x  =  /(x,u),  then  the  OSS  property  amounts  to  a  version 
of  the  input  to  state  stability  property  for  this  first  subsystem,  but  only  with  respect  to  the  signals  so 
generated. 

In  feedback  control,  the  concept  of  passivity  has  been  widely  used.  System  (1)  with  m  —  p  is  said 
to  be  strictly  passive  is  there  exist  a  continuous  nonnegative  function  V  :  En  E>0 ,  called  a  storage 
function,  and  a  positive  definite  function  a,  called  the  dissipation  rate,  such  that  for  any  f  G  Mn  and 
any  input  u,  t 

V(x(t,t,u)) -V(0  <  -  f  a(\x(s,£,u)\)ds+  [  y{s,£,u)u(s)  ds  (8) 

Jo  Jo 

for  all  t  >  0.  If,  as  usually  supposed,  V  is  differentiable,  positive  definite,  and  proper,  and  if  a  in  (8)  is 
also  proper,  then  V  is  an  IOSS-Lyapunov  function,  and  the  system  is  IOSS.  (If  V(x(t))  does  not  have  a 
derivative  along  trajectories,  one  obtains  an  integral  equation  version  of  (7);  it  is  possible  to  show  that 
the  existence  of  a  V  with  such  a  property  also  implies  that  the  system  is  IOSS.) 

There  is  a  remarkable  relationship  between  IOSS  and  the  possibility  of  estimating  the  norms  of  states 
“on-line”.  By  a  (one-sided)  “state-norm  estimator”  for  the  system  (1)  we  mean  a  system 

z  =  g(z,u,y )  (9) 

whose  inputs  are  pairs  (u(t),  j/(t))  consisting  of  inputs  and  outputs  of  (1),  which  is  itself  ISS  (with  inputs 
( u,y )),  and  which  has  the  property  that,  for  some  function  p  of  class  K.  and  a  function  (3  of  class  ICC, 
for  each  initial  states  f  and  C  for  (1)  and  (9)  respectively,  and  each  input  u(-), 

|z(L£.«)l  <  0(|£|  +  |C|,t)  +P(lz(t>C.«.J/€,t<)|) 
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for  alH  £  [0,  fmax)*  (That  is,  the  z  equation  provides  an  upper  bound  p{z(t))  on  the  true  state  x(t):  with 
an  error  which  is  initially  small  if  £  and  £  are  small,  and  which  in  any  case  decays  to  zero  as  t  -»  oo.) 
We  conjecture  that  the  existence  of  a  norm-estimator  for  the  system  is  equivalent  to  the  IOSS  property; 
for  systems  with  no  inputs,  this  was  already  proved  in  the  recent  paper  [14], 

Finally,  a  few  details  about  observers.  A  (full-order  state)  observer  for  the  system  (1)  is  a  system 
defined  by  equations  z  =  g(z,v,w)  evolving  in  the  same  space  Rn  as  (1),  driven  by  inputs  v  and  w  of 
dimensions  equal  to  the  dimension  of  the  input  and  output  value  spaces  of  (1),  respectively,  and  so  that 
the  following  properties  hold:  There  exist  functions  /?  £  JCC,  and  71,72  €  /C  such  that,  for  each  initial 
states  £  and  £  of  the  composite  system  consisting  of  (1)  and 

z  =  g{z,u  +  ud,y  +  y<i)  (10) 

and  each  (measurable  locally  essentially  bounded)  inputs  u,ttd,2/d5  if  [0,£max)  is  the  maximal  interval  of 
existence  of  x(f)  =  x(t,£,u)  then  the  solution  z(t)  of  (10)  with  z(0)  =  £,  y(t)  —  h(x(t)))  and  the  same 
w,Ud,r/d  is  also  defined  on  [0,  tmax)  and  there  holds  on  [0,  tmax)  the  estimate 

|x(t)-2(f)|  <  max  {/?(|£  -  Cl ,  t),  71  (||«d|[o,t]||) ,  72  (|M[o,t]||)}  • 

It  is  easy  to  see  that  the  equations  of  any  observer  (10)  must  have  the  (“output  injection”)  form 

z  —  /(z,  u  +  Ud)  4*  L(z ,  u  +  ud)  y  -  h(z)  +  yd) 

where  the  vector  field  L  satisfies  that  L(a,  6,0)  =  0  for  all  a,  6. 

Let’s  call  a  system  (1)  incrementally  (Lipschitz)  input/ output-to- state  stable  (i-IOSS)  if  there  exists 
some  /?  £  ICC  and  71,72  €  K  such  that,  for  every  two  initial  states  £1  and  £2,  and  any  two  controls  ui 
and  U2, 

k(f»fi>«i) -x(t,&,U2)|  <  max {/?(|&  -&|,i)7i  (||(«i  -  «2)|[o,t]||)  ,72  (||G/«i, «i  ~  J/C2,«2)l[o,t]||)} 
for  all  t  in  the  common  domain  of  definition. 

One  can  then  see  that  if  an  observer  for  (1)  exists,  the  system  is  i-IOSS,  and  thus  also  IOSS.  In 
general,  however,  the  IOSS  property  does  not  guarantee  the  existence  of  an  observer.  To  see  this, 
it  suffices  to  give  an  example  of  a  system  that  is  IOSS  but  not  i-IOSS:  the  one-dimensional  system 
x  =  u,y  =  x2  is  one  such  example. 

The  output-injection  form  of  the  observer,  in  the  special  case  x  =  0,  u  =  0,  and  yd  =  y,  allows 
concluding  that  the  system  x  =  f(x,u)  +  L(x,  w,  —  /i(z))  is  ISS,  which  generalizes  the  classical  notion  of 
output  injection  for  linear  systems. 

2.3  Stability  Notions:  IOS 

We  complete  the  discussion  of  stability  notions  by  returning  to  yet  another  concept  introduced 
in  the  Pi’s  by  now  well-known  1989  ISS  paper:  input  to  output  stability  (IOS,  for  short).  In 
informal  terms,  this  means  that  the  output  (instead  of  the  full  state)  must  be  eventually  small, 
no  matter  what  the  initial  When  there  are  no  inputs,  one  has  a  generalization  of  the  classical 
concept  of  partial  stability.  In  general,  this  is  the  appropriate  notion  to  study  in  the  context  of 
regulation  problems.  It  turns  out,  however,  that  the  notion  of  IOS  given  in  the  1989  paper  is 
not  exactly  the  appropriate  one  for  modeling  the  situation  typical  in  regulation  or  in  robust  and 
adaptive  control,  where  a  condition  of  boundedness  of  internal  variables  is  required  in  addition 
to  asking  that  outputs  become  small.  The  concept  must  be  modified  somewhat.  There  is  a 
fairly  obvious  connection  between  the  various  concepts  introduced  until  now:  a  system  is  ISS 
if  and  only  if  it  is  both  IOSS  and  IOS.  This  fact  generalizes  the  linear  systems  theory  result 
“internal  stability  is  equivalent  to  detectability  plus  external  stability”  and  its  proof  follows  by 
routine  arguments. 
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A  long-term  objective  is  to  obtain  necessary  and  sufficient  characterizations  of  the  solvabil¬ 
ity  of  complete  regulation  problems  in  this  framework.  In  other  approaches,  stability  with  no 
disturbances  is  considered  as  a  totally  separate  property  from  regulation  (in  particular,  prob¬ 
lems  such  as  tracking);  in  the  current  context,  these  different  aspects  are  taken  into  account 
simultaneously. 

IOS:  Some  technical  details* 

Recall  the  GS  property  defined  in  page  6.  Note  that  for  a  system  with  no  controls, 

x  =  f(x),  y  =  h(x), 

the  GS  property  reduces  to  a  “bounded  state”  estimate: 

|*(t,0|  <  o-(KI),  Vt>  0,V^6En 

This  property  amounts  to  (neutral)  stability  plus  what  is  sometimes  called  “uniform  boundedness” . 

The  main  property  that  we  are  now  considering  is  as  follows.:  The  system  (1)  is  input  to  output 
stable  (IOS)  if  it  is  GS  and,  in  addition,  there  exist  a  AC£-function  and  a  AC-function  -y  such  that 

l»(*,f,«)|</?(KI,<)  +  7(INI),  vt>o,  (ii) 

holds  for  all  u  and  all  £  G  Mn.  For  an  autonomous  system,  we  say  simply  output  stable  (OS).  That  is, 
such  a  system  is  OS  if  it  satisfies  an  a  bounded-state  estimate  and  there  is  some  /C£-function  /3  such 
that 

\vM\<m\>t)  v*>0, 

holds  for  all  £  €  Rn.  Clearly,  if  system  (1)  is  IOS,  then  the  associated  0-input  system  x  =  f(x,  0)  is  OS. 

When  h  is  the  identity,  IOS  and  ISS  coincide,  and  OS  is  exactly  the  same  as  global  asymptotic 
stability.  Observe  that  the  GS  property  is  redundant  in  that  case,  as  it  is  obviously  implied  by  the  decay 
estimate  (11),  letting  a  =  min{/?(-,0),7}.  We  can  prove  that  a  system  is  IOS  if  and  only  if  it  is  GS  and 
there  exist  ai,a2  G  fC0 G,  X  G  and  0:3  G  /C£,  such  that  for  each  r  >  0,  there  is  a  smooth  function  V 
so  that: 

•  Oi(|fc(0|)  <  V(Z)  <  aaCIfl)  for  all  *  €  R"; 

•  DV(£)f(£,n)  <  -a3(V(f),|f|)  for  all  f  such  that  V(f)  >  x(r)  and  a11  H  <  r- 

This  Lyapunov  condition  is  far  more  complicated  than  one  would  like.  It  says  in  essence  that  IOS  is 
equivalent,  for  each  bound  on  controls,  to  the  existence  of  an  Lyapunov-like  function  V  which  vanishes 
only  when  the  output  vanishes,  and  whose  derivative  along  trajectories  is  negative  (unless  either  the 
function  is  already  zero  or  the  current  input  is  large).  Moreover,  the  rate  of  decay  of  V ( x{t ))  depends  on 
the  state  and  on  the  value  of  V(x(t))  (the  main  role  of  a3  is  to  allow  for  slower  convergence  if  V(x(t)) 
is  very  small  or  if  x(t)  is  very  large;  the  inequality  can  be  restated  in  various  alternative  ways).  We  do 
not  yet  know  if  IOS  implies  the  existence  of  one  V  (independent  of  the  input  level  r)  with  the  stated 
properties. 

For  the  case  of  no  inputs,  the  above  applied  with  r  =  0  provides  the  following,  apparently  new,  result 
for  the  OS  property:  OS  is  equivalent  to  the  boundedness  condition  plus  the  existence  of  ai,a2  E  IC0 0) 
and  <23  €  /C£,  and  a  smooth  function  V,  so  that: 

•  ai(|/i(OI)  <  V(0  <  a2(|f|)  for  all  £  G  Rn; 

•  DV(0f( 0  <  -o3(V(0,  l£l)  for  all  £  G  Rn. 

For  this  case,  no  inputs,  the  above  Lyapunov  property  had  already  appeared  in  the  literature,  but  only 
as  a  sufficient  condition. 


*this  subsection  can  be  skipped  without  loss  of  continuity. 
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2.4  Discontinuous  Feedback 

A  longstanding  open  question  in  nonlinear  control  theory  concerns  the  relationship  between 
asymptotic  controllability  to  the  origin  in  R"  of  a  nonlinear  system  (1)  by  an  “open  loop” 
control  u  :  [0,  +oo)  — >  Km  and  the  existence  of  a  stabilizer,  that  is  to  say  a  system  which 
processes  the  outputs  y  =  h(x)  and  provides  appropriate  control  signals  u  which  drive  the  state 
to  zero.  A  particular  version  of  this  problem  is  the  (memoryless)  state  feedback  problem:  find 
a  map  k  :  E"  ->  Rm  which  stabilizes  trajectories  of  the  system 

x  =  f(x,k{x ))  (12) 

with  respect  to  the  origin.  (As  earlier  remarked,  objectives  other  than  state  stabilization,  such 
as  tracking  and  disturbance  rejection,  output  feedback,  or  adaptive  and  robust  variants,  are  of 
even  more  interest,  and  stabilization  is  merely  the  first  step  in  the  development  of  a  general 
theory.) 

For  the  special  case  of  linear  control  systems  x  =  Ax  +  Bu,  this  relationship  between  open- 
loop  and  closed-loop  notions  is  well  understood:  asymptotic  controllability  is  equivalent  to 
the  existence  of  a  continuous  (even  linear)  stabilizing  feedback  law.  But  it  is  well-known  that 
continuous  feedback  laws  may  fail  to  exist  even  for  simple  asymptotically  controllable  nonlinear 
systems.  This  is  especially  easy  to  see  for  one-dimensional  (m  =  n  =  1)  systems  (1):  in  that 
case  asymptotic  controllability  is  equivalent  to  the  property  “for  each  x  /  0  there  is  some  value 
u  so  that  xf(x,u)  <  0”,  but  it  is  easy  to  construct  examples  of  functions  /,  even  analytic,  for 
which  this  property  is  satisfied  but  for  which  no  possible  continuous  section  k  :  R  \  {0}  E 
exists  so  that  xf(x,k(x))  <  0  for  all  nonzero  x. 

These  negative  results  led  to  the  search  for  feedback  laws  which  are  not  necessarily  of  the 
form  u  —  k(x),  k  a  continuous  function.  One  possible  approach  is  to  look  for  dynamical 
feedback  laws,  where  additional  “memory”  variables  are  introduced  into  a  controller,  and  as 
a  very  special  case,  time- varying  (even  periodic)  continuous  feedback  u  =  k(t,x).  Such  time- 
varying  laws  were  shown  by  the  Pi  and  Sussmann  in  1980  to  be  always  possible  in  the  case 
of  one-dimensional  systems,  and  in  the  major  work  of  Coron  in  1992  it  was  shown  that  they 
are  also  always  possible  when  the  original  system  is  completely  controllable  and  has  “no  drift”, 
meaning  essentially  that  /(x,0)  =  0  for  all  states  (see  also  [8]  for  an  alternative  proof  of 
the  time- varying  result  for  analytic  systems).  However,  for  the  general  case  of  asymptotically 
controllable  systems  with  drift,  no  dynamic  or  time-varying  continuous  solutions  are  known. 
Because  of  this,  and  also  because  it  is  in  any  case  the  obvious  natural  mathematical  question, 
one  may  ask  about  the  existence  of  discontinuous  feedback  laws  u  =  k(x).  Such  feedbacks  are 
often  obtained  when  solving  optimal- control  problems,  for  example,  so  it  is  interesting  to  search 
for  general  theorems  insuring  their  existence.  Unfortunately,  allowing  nonregular  feedback  leads 
to  an  immediate  difficulty:  how  should  one  define  the  meaning  of  solution  x( •)  of  the  differential 
equation  (12)  with  discontinuous  right-hand  side? 

One  of  the  best-known  candidates  for  the  concept  of  solution  of  (12)  is  that  of  a  Filippov 
solution ,  which  is  defined  as  the  solution  of  a  certain  differential  inclusion  with  multivalued  right- 
hand  side  which  is  built  from  /(x,  k(x)).  However,  it  follows  from  recent  results  of  Coron,  Rosier, 
and  Ryan,  that  the  existence  of  a  discontinuous  stabilizing  feedback  in  the  Filippov  sense  implies 
the  same  Brockett  necessary  conditions  as  the  existence  of  a  continuous  stabilizing  feedback 
does.  Moreover,  the  existence  of  a  stabilizing  feedback  in  the  Filippov  sense  is  equivalent  to  the 
existence  of  a  continuous  stabilizing  one,  in  the  case  of  systems  affine  in  controls.  In  conclusion, 
there  is  no  hope  of  obtaining  general  results  if  one  insists  on  the  use  of  Filippov  solutions. 


2  DETAILS 


12 


In  the  major  recent  paper  [13],  done  in  collaboration  with  Yuri  Ledyaev  (who  is  at  Rutgers 
as  a  long-term  visitor)  and  Clarke  and  Subbotin,  we  developed  a  concept  of  solution  of  (12) 
for  arbitrary  feedback  k(x)  which  has  (a)  a  clear  and  reasonable  physical  meaning  (perhaps 
even  more  so  than  the  definitions  derived  from  differential  inclusions),  and  (b)  allows  proving 
the  desired  general  theorem.  Our  notion  is  borrowed  from  the  theory  of  positional  differential 
games,  and  it  was  systematically  studied  in  that  context  by  Krasovskii  and  Subbotin. 

There  had  been  several  other  papers  dealing  with  rather  general  theorems  on  discontinu¬ 
ous  stabilization.  One  of  the  best  known  is  Sussmann’s  work,  which  provided  piecewise  ana¬ 
lytic  feedback  laws  for  analytic  systems  which  satisfy  controllability  conditions.  The  definition 
of  “feedback”  given  in  that  paper  involves  a  specification  of  “exit  rules”  for  certain  lower¬ 
dimensional  submanifolds,  and  these  cannot  be  expressed  in  terms  of  a  true  feedback  law  (even 
in  the  sense  of  [13]).  Sampling  is  a  strategy  commonly  used  in  digital  control.  Sampling  is 
not  true  feedback,  in  that  one  typically  uses  a  fixed  sampling  rate,  or  perhaps  a  predetermined 
sampling  schedule,  and  intersample  behavior  is  not  accounted  for.  One  may  interpret  our  so¬ 
lutions  as  representing  the  behavior  of  sampling,  with  a  fixed  feedback  law  being  used,  as  the 
sampling  periods  tend  to  zero  -  indeed,  such  a  speedup  of  sampling  is  essential  as  we  approach 
the  target  state,  to  avoid  an  overshoot  during  the  sampling  interval,  as  well  as  far  from  the 
target,  due  to  possible  explosion  times  in  the  dynamics. 

This  work  has  opened  a  whole  new  research  direction,  having  to  do  with  the  solution  of 
all  control  design  problems  in  the  new  framework.  More  recent  work,  e.g.  [29]  and  continuing 
work,  has  ellucidated  the  relationship  to  effects  of  observation  and  actuator  errors. 

Nonsmooth  Feedback:  Some  technical  details* 

We  call  any  infinite  sequence  7r  =  {tj}f>0  consisting  of  numbers  0  =  t0  <  h  <  t2  <  . . .  with  limi-n*,  L  = 
oo  a  partition  (of  [0,  +oo)),  and  the  number  d(7r)  :=  supi>0(t!+i  —  tf)  its  diameter.  Let’s  call  any 
locally  bounded  (for  each  compact,  image  is  relatively  compact)  function  k  :  E"  -»■  Km  a  feedback. 
curv-97prop-control  We  next  define  the  trajectory  associated  to  a  feedback  k(x)  and  any  given  partition 
7 r  as  the  solution  obtained  by  means  of  the  following  procedure:  on  each  interval  [L, t,+i],  the  initial 
state  is  measured,  Ui  =  k(x(L))  is  computed,  and  then  the  constant  control  u  =  ul  is  applied  until  time 
ti+\ ,  when  a  new  measurement  is  taken:  Given  an  xo  €  Kn ,  for  each  i,  i  =  0, 1, 2, . . .,  recursively  solve 

x(t)  =  f{x(t),k(x(ti))) ,  t  €  [ti,ti+i]  (13) 

using  as  initial  value  x(L)  the  endpoint  of  the  solution  on  the  preceding  interval  (and  starting  with 
x(f0)  =  xo)-  The  7r -trajectory  of  (12)  starting  from  x0  is  the  function  x(-)  thus  obtained.  Observe  that 
this  solution  may  fail  to  be  defined  on  all  of  [0,  +oo),  because  of  possible  finite  escape  times  in  one  of 
the  intervals,  in  which  case  we  only  have  a  trajectory  defined  on  some  maximal  interval.  In  our  results, 
however,  the  construction  provides  a  feedback  for  which  solutions  are  globally  defined;  we  say  in  that 
case  that  the  trajectory  is  well-defined. 

We  next  define  the  meaning  of  (globally)  stabilizing  feedback.  This  is  a  feedback  law  which,  for  fast 
enough  sampling,  drives  all  states  asymptotically  to  the  origin  and  with  small  overshoot.  Of  course,  since 
sampling  is  involved,  when  near  the  origin  it  is  impossible  to  guarantee  arbitrarily  small  displacements 
unless  a  faster  sampling  rate  is  used,  and,  for  technical  reasons  (for  instance,  due  to  the  existence  of 
possible  explosion  times),  one  might  also  need  to  sample  faster  for  large  states.  Thus  the  sampling  rate 
needed  may  depend  on  the  accuracy  desired  when  controlling  to  zero  as  well  as  on  the  rough  size  of  the 
initial  states,  and  this  fact  is  captured  in  the  following  definition.  The  feedback  k  :  Rn  — >  R,n  is  said 
to  s-stabilize  the  system  (1)  if  for  each  pair  0  <  r  <  R  there  exist  M  =  M(R)  >  0,  S  =  S(r,  R)  >  0, 
and  T  =  T(r,R)  >0  such  that,  for  every  partition  n  with  d( n)  <6  and  for  any  initial  state  x0  such 


"this  subsection  can  be  skipped  without  loss  of  continuity. 
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that  \xQ\  <  i?,  the  7r-trajectory  x(-)  of  (12)  starting  from  x0  is  well-defined  and  it  holds  that:  (uniform 
attractiveness:)  \x(t)\  <  r  Vt  >  T;  (overshoot  boundedness:)  \x(t)\  <  M(R)  Vt  >  0;  and  (Lyapunov 
stability:)  lim^o  M  (I?)  =  0.  If  a  continuous  feedback  k  stabilizes  the  system  (1)  in  the  usual  sense 
(namely,  it  makes  the  origin  of  (12)  globally  asymptotically  stable),  then  it  also  s-stabilizes.  In  this 
sense,  the  present  notion  generalizes  the  classical  notion  of  stabilization. 

The  system  (1)  is  asymptotically  controllable  if:  (attractiveness:)  for  each  xo  £  Mn  there  exists  some 
control  u  such  that  the  trajectory  x(t)  =  x(t;x0iu)  is  defined  for  all  t  >  0  and  x(t)  0  as  t  -boo; 
(Lyapunov  stability:)  for  each  e  >  0  there  exists  8  >  0  such  that  for  each  xQ  £  Rn  with  \xq\  <8  there  is 
a  control  u  as  above  such  that  |x(*)|  <  £  for  all  t  >  0;  and  (bounded  controls:)  there  are  a  neighborhood 

V  of  0  in  Rn,  and  a  compact  subset  U  of  Mm  such  that,  if  the  initial  state  a?o  above  satisfies  also  xo  6  V , 
then  the  control  as  stated  can  be  chosen  with  u(t)  £  U  for  almost  all  t. 

Our  main  result  in  [13]  is  as  follows:  A  system  is  asymptotically  controllable  if  and  only  if  it  admits 
an  s-stabilizing  feedback .  One  implication  is  easy:  existence  of  an  s-stabilizing  feedback  is  easily  seen  to 
imply  asymptotic  controllability.  The  interesting  implication  is  the  converse,  namely  the  construction 
of  the  feedback  law.  The  main  ingredients  in  this  construction  are:  (a)  the  notion  of  control-Lyapunov 
function,  (b)  methods  of  nonsmooth  analysis,  and  (c)  techniques  from  positional  differential  games. 

Given  a  function  V  :  En  — >  E  and  a  vector  v  £  Rn ,  the  lower  directional  derivative  of  V  in  the 
direction  of  v  is  DV(x;v)  :=  limsupt^0>v/_yt;  j  4-  tv1)  -  y(z)).  The  function  v  DV(x;v)  is  lower 
semicontinuous.  For  a  set  F  C  En,  coF  denotes  its  convex  hull.  A  control-Lyapunov  pair  for  the 
system  (1)  consists  of  two  continuous  functions  V,  W  :  En  — >  R>o  such  that  the  following  properties 
hold:  (positive  definiteness:)  F(x)  >0  and  W(x)  >0  for  all  x  ±  0,  and  V(0)  =  0;  (properness:)  the 
set  {x  |  V^z)  <  fi]  is  bounded  for  each  f}\  and  (infinitesimal  decrease:)  for  each  bounded  subset  G  CIn 
there  is  some  compact  subset  U  C  Rm  such  that 

min  DV{x\v)  <  —W(x)  (14) 

v€co  f(xyU) 

for  every  x  £  G.  If  V"  is  part  of  a  control-Lyapunov  pair  (F,  W)^  it  is  a  control-Lyapunov  function  ( elf ). 

It  was  shown  in  a  paper  by  the  PI  in  1983  that  asymptotic  controllability  is  equivalent  to  the  existence 

of  a  pair  of  functions  (V,W)  which  satisfy  the  properties  given  above,  except  that  the  infinitesimal 

decrease  property  is  expressed  in  an  apparently  weaker  fashion,  namely,  by  means  of  derivatives  along 
trajectories  (corresponding  to  relaxed  controls).  In  [24]  it  was  observed  that  in  fact  one  can  reformulate 
the  definition  in  the  above  terms,  so  a  system  is  asymptotically  controllable  if  and  only  if  it  admits  a 
control-Lyapunov  function.  Of  course,  when  the  function  V  is  smooth,  condition  (14)  can  be  written  in 
the  more  familiar  form  found  in  the  literature,  namely: 

min(VV'(a:),/(a:,u))  <  —  W(x).  (15) 

u£U 

In  contrast  to  the  situation  with  stability  of  (non-controlled)  differential  equations,  a  system  may  be 
asymptotically  controllable  system  and  yet  there  may  not  exist  any  possible  smooth  elf  V.  In  other  words, 
there  is  no  analogue  of  the  classical  theorems  due  to  Massera  and  Kurzweil.  This  issue  is  intimately 
related  to  that  of  existence  of  continuous  feedback,  via  Artstein’s  Theorem,  which  asserts  that  existence 
of  a  differentiable  V  is  equivalent,  for  systems  affine  in  controls,  to  there  being  a  stabilizing  regular 
feedback.  Nevertheless,  it  is  possible  to  reinterpret  the  condition  (15)  in  such  a  manner  that  relation  (15) 
does  hold  in  general,  namely  by  using  a  suitable  generalization  of  the  gradient.  Specifically,  as  remarked 
in  [13],  we  may  use  the  proximal  subgradients  of  V  at  x  instead  of  W(x),  replacing  (15)  by: 

min(£,  f(x,u))  <  -W(x)  for  every  £  £  dpV(x) ,  (16) 

ueu 

where  £  and  dpV(x )  are  the  proximal  subgradients  and  the  sub  differential,  respectively,  of  the  function 

V  at  the  point  x.  The  use  of  proximal  subgradients  as  substitutes  for  the  gradient  for  a  nondifferentiable 
function  plays  a  central  role  in  our  construction  of  feedback. 

The  relation  (16)  says  that  V  is  a  “proximal  supersolution”  of  the  corresponding  Hamilton- Jacobi 
equation,  which  is  known  to  be  equivalent  to  the  statement  that  locally  V  is  a  viscosity  supersolution  of 
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the  same  equation.  On  the  other  hand,  the  relation  (14)  says  that  V  is  an  upper  minimax  solution  of  the 
same  equation.  The  coincidence  of  these  two  solution  concepts  reflects  the  deep  and  intrinsic  connection 
between  invariance  properties  of  function  V  with  respect  to  trajectories  of  the  control  system  (1)  and 
the  characterization  of  these  properties  in  terms  of  proximal  subgradients  of  V.  The  proof  of  the  main 
result  in  [13]  relies  upon  the  smoothing  of  a  elf  V  on  compacts;  this  smoothing  is  not  itself  a  elf,  but  it 
allows  finding  suitable  proximal  subgradients,  which  in  turn  provide  the  directions  for  control. 


2.5  Universal  Controls 

A  long-standing  direction  of  the  Pi’s  research  concerns  the  study  of  transversality-type  theorems 
which  assert  the  existence  of  controls  with  particularly  useful  characteristics.  In  the  case  of 
inputs  universal  for  observability,  the  study  leads  to  the  deep  understanding  of  relationships 
between  different  types  of  observation  spaces  and  the  existence  of  i/o  equations,  which  in  turn 
give  rise  to  identification  procedures;  in  the  case  of  universality  for  controllability,  one  finds 
algorithms  for  path  planning.  We  briefly  describe  some  of  these  ideas  next. 

The  Pi’s  presentation  and  paper  at  the  ICM  [17]  dealt  with  observation  spaces,  and  in 
particular  with  the  applications  of  these  to  the  study  of  the  equivalence  between  i/o  high  order 
differential  equations  E  (w(t),w'(t),w"(t), . . .  ,w(r)(0)  =  0,  where  w(-)  =  ( u{-),y( •))  represent 
the  i/o  pairs  of  the  system,  and  state-space  representations.  In  the  linear  case,  such  represen¬ 
tations  form  the  basis  of  much  of  adaptive  control  and  identification  theory;  and  for  nonlinear 
systems,  (the  discrete-time  versions  of)  our  results  have  also  been  applied  to  identification  al¬ 
gorithms.  Central  to  this  work  is  the  analysis  of  observation  spaces,  in  their  different  variants. 
These  are  essentially  spaces  spanned  by  derivatives  of  functions  on  states  induced  by  inputs, 
and  different  algebraic  structures  can  be  introduced,  thus  obtaining  algebras  and  fields  which 
characterize  systems  properties  (for  instance,  states  of  the  system  appear  as  the  prime  spec¬ 
trum,  in  the  Gelf’and-Grothendieck  style,  of  rings  of  observables;  see  [17]).  Most  results  about 
observation  spaces  depend  on  the  existence  of  inputs  universal  for  distinguishability.  We  wish 
to  continue  research  in  this  area,  and  in  particular  we  want  to  write  a  detailed  technical  paper 
showing  that  there  axe  controls  which  are  universal  with  respect  to  the  class  of  all  analytic 
(finite  dimensional)  systems  (only  a  brief  sketch  of  this  fact  appeared  in  a  recent  conference 
proceedings). 

A  dual  version  of  the  universality  results  allowed  us  to  obtain  a  result  insuring  the  existence 
and  genericity  of  universal  nonsingular  controls;  this  led  us  in  turn,  in  [8],  to  the  development 
of  a  simple  numerical  technique  for  the  steering  of  arbitrary  analytic  systems  with  no  drift.  It 
is  based  on  the  generation  of  “nonsingular  loops”  which  allow  linearized  controllability  along 
suitable  trajectories.  Once  such  loops  are  available,  it  is  possible  to  employ  standard  Newton 
or  steepest  descent  methods,  as  classically  done  in  numerical  control.  During  this  last  year 
(1996),  we  worked  with  a  graduate  student  at  Rutgers  (Master’s  thesis  by  R.  Woolley)  in 
the  development  of  efficient  codes  for  implementing  the  method  in  [8].  The  resulting  code, 
obtainable  by  e-mail  from  the  PI,  is  still  preliminary,  but  runs  fast  on  simple  examples.  The 
next  section  briefly  discusses  technically  the  method. 

As  a  final  subtopic,  we  remark  that  the  existence  of  universal  controls  for  nonsingularity 
of  discrete  time  systems  is  still  not  completely  resolved,  but  in  current  work  with  F.  Wirth  we 
have  succeeded  in  proving  the  main  technical  results,  and  are  now  looking  into  the  estimation 
of  effective  bounds  for  input  lengths. 
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Nonsingular  Loop  Approach:  Some  technical  details* 

The  problem  of  interest  concerns,  for  any  given  initial  and  target  states  f0  and  =  0  in  Kn ,  numerically 
finding  a  time  T  >  0  and  a  control  u  which  (approximately)  steers  to  in  (1),  i.e.  x(T,£0,u)  ss  £F. 
For  now,  we  assume  that  the  right-hand  side  /  is  continuously  differentiable  (later,  analytic). ’classical 
numerical  techniques  for  this  problem  are  based  on  variations  of  steepest  descent.  The  basic  idea  is  to 
start  with  a  guess  of  a  control,  say  u  :  [0,  T]  Mm ,  and  to  improve  iteratively  on  this  initial  guess.  More 
precisely,  let  x  =  x(-,£o,u).  If  the  obtained  final  state  x(T)  is  already  zero,  or  is  sufficiently  near  zero, 
the  problem  has  been  solved.  Otherwise,  we  look  for  a  perturbation  Au  so  that  the  new  control  u  +  Alt 
brings  us  closer  to  our  goal  of  steering  to  the  origin.  The  various  techniques  differ  on  the  choice  of 
the  perturbation;  in  particular,  two  possibilities  are  Newton’s  method  and  steepest  descent. 

As  an  illustration,  take  Newton’s  method.  Denote,  for  any  fixed  initial  state  £0,  a(u)  x(T,f0,u), 
thought  of  as  a  partially  defined  map  from  £™(0,  T)  into  R".  This  is  a  continuously  differentiable  map 
so  expanding  to  first  order  there  results  a(u  +  v)  =  a(u )  +  a*[u](u)  -f  o(v)  for  any  other  control  v  so 
that  a(u  +  v )  is  defined,  where  we  use  “*”  as  a  subscript  to  denote  differentials.  If  we  can  now  pick  v  so 
that 

a»[u](u)  = -a(u),  (17) 

then  for  small  enough  h  >  0  real, 

a(u  +  hv)  =  (1  -  h)a(u)  +  o{h)  (18) 

will  be  smaller  than  the  state  a(u)  reached  with  the  initial  guess  control  u.  In  other  words,  the  choice 
of  perturbation  is  Au  :=  hv,  0  <  h  -C  1.  It  remains  to  solve  equation  (17)  for  v.  The  operator 
L-v  i->  a»[u](u)  is  the  one  corresponding  to  the  solution  of  the  variational  equation 

i  =  A{t)z  +  B{t)v  z(0)  =  0,  (19) 

where  A(f)  :=  f £(x(t),u(f))  and  B(t)  :=  $fc(x(t),u(t))  for  each  t,  that  is,  Lv  =  /0T  <t>(T,s)B(s)v(s)  ds, 
where  $  denotes  the  fundamental  solution  associated  to  X  -  A(t)X.  The  operator  L  maps  £™(0,T) 
mto  Rn ,  and  it  is  onto  when  (19)  is  a  controllable  linear  system  on  the  interval  [0,  T],  that  is,  when  u  is  a 
control  nonsingular  for  £0  relative  to  the  system  (1).  In  other  words,  ontoness  of  L  =  a*[u]  is  equivalent 
to  first-order  controllability  of  the  original  nonlinear  system  along  the  trajectory  corresponding  to  the 
initial  state  £o  an<f  the  control  u.  The  main  point  of  the  paper  [8]  was  to  show  that  it  is  not  difficult  to 
generate  useful  nonsingular  controls  for  systems  with  no  drift. 

Assuming  nonsingularity,  there  exist  then  many  solutions  to  (17).  It  is  reasonable  to  pick  the  least 
squares  solution,  that  is  the  unique  solution  of  minimum  norm,  v  :=  -L*a(u)  where  L*  denotes 
the  pseudoinverse  operator.  The  technique  sketched  above  is  well-known  in  numerical  control.  For 
instance,  the  derivation  in  pages  222-223  of  the  classical  text  by  Bryson  and  Ho,  when  applied  to  solving 
the  optimal  control  problem  having  the  trivial  cost  criterion  J(u)  =  0  and  subject  to  the  final  state 
constraints  x  =  ip(x)  =  0,  results  in  the  above  formula,  and  is  derived  in  the  same  manner  as  here.  It 
is  also  possible  to  combine  the  technique  with  line  searches  over  the  scalar  parameter  h  or,  even  more 
efficiently  in  practice,  with  conjugate  gradient  approaches. 

A  result  in  [8]  provides  a  convergence  proof  of  an  iterative  procedure  based  on  this  idea,  for  analytic 
systems  with  no  drift  x  =  G(x)u.  In  order  to  apply  the  numerical  technique  described,  one  needs  to 
find  a  control  u  which  leads  to  nonsingular  loops:  u  is  nonsingular  for  every  state  x  in  a  given  ball 
B,  and  x(T,x,u )  =  x  for  all  such  x .  It  is  shown  there  that  for  analytic  systems  that  have  the  strong 
accessibility  property,  controls  which  are  generic  —in  a  Whitney  topology  sense—  are  nonsingular  for  all 
states.  (For  analytic  systems  without  drift,  Chow’s  Theorem  states  that  the  strong  accessibility  property 
is  equivalent  to  complete  controllability.)  Starting  from  such  a  control  w,  defined  on  an  interval  [0,T/2], 
one  may  now  consider  the  control  u  on  [0,  T)  which  equals  u  on  [0,  T/2]  and  is  then  followed  by  the 
antisymmetric  extension:  u(t)  =  -c o(T  -t),te  (T/2.T].  This  it  is  as  needed:  nonsingularity  is  due  to 
the  fact  that  if  the  restriction  of  a  control  to  an  initial  subinterval  is  nonsingular  for  the  initial  state,  the 

*this  subsection  can  be  skipped  without  loss  of  continuity. 
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whole  control  is,  and  the  loop  property  is  an  easy  consequence  of  the  fact  that  there  is  no  drift  (so  the 
control  appears  linearly).  In  practice,  we  have  implemented  instead  a  finite  Fourier  series  with  random 
coefficients:  u(t)  =  J2k=i  a*  sin  which  automatically  satisfies  the  antisymmetry  property  on  the 
time  interval  [0,27r].  As  an  extremely  simple  example,  consider  the  classical  “knife  edge”  or  “unicycle” 
problem  with  equations 


x  —  U\  cos  6 
y  —  u\  sin  9 
6  =  V>2  - 

We  use  a  two-term  Fourier  series  with  T  =  2n  and  randomly  generated  uniform  [-1,1]  coefficients  to 
obtain  (generically)  a  nonsingular  control.  The  figure  shows  a  typical  path  and  the  controls  used. 


Obviously,  this  is  far  from  a  best  possible,  or  perhaps  even  desirable,  path.  But  it  is  striking  that 
the  algorithm  is  able  to  find  a  solution  (and  is  guaranteed  to  do  so)  without  any  built-in  knowledge 
whatsoever  of  the  Lie  algebraic  structure;  only  the  equations  of  motions  are  needed  (for  an  internal 
model)  and  the  form  of  the  variational  equation. 
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